feat(workflows): let agents set up an email sending domain via mcp - #343

Problem

Feature flags: workflows-email-domain-agent-setup. Default off; this handoff does not activate them.

• An agent using the PostHog MCP cannot set up a Workflows email sender: the create, verify and DNS tools are all disabled.
• Enabling them alone would not work. Personal API keys and MCP OAuth tokens get "This action does not support personal API key access" on verify, email update and both Domain Connect actions, because none of them is in the viewset's scope lists.
• The endpoints also declare no schemas, so generated tools would get untyped or wrong inputs.
• This is PR B of the email domain setup redesign. PR A (the wizard UI and a read-only status endpoint) ships in parallel, not stacked.

Fork handoff for source #109703. Modernized head 8cbc65746ebc271925c464070c2bd165bb23cf19. This draft prepares review and recreation in the upstream repository.

Changes

Start with posthog/api/integration.py and posthog/api/integration_domain_serializers.py.

• Agents can now take a domain from "add a sender" to "verified" over MCP, behind the workflows-email-domain-agent-setup flag. The backend checks the same flag, so turning it on enables the tools and the API access they need together:

• Tool descriptions carry the flow: create, get records, then either hand the person the Domain Connect URL or add records with the agent's own DNS tools without a second _dmarc, then verify until success.
• A new setting-up-an-email-domain skill teaches that job, including which records to merge (SPF, existing DMARC) instead of duplicating.
• The skill relies on #109695: the Domain Connect URL leaves out the DMARC record when the domain already has one. It is merged, and this branch includes it.
• The skill and tools treat the MAIL FROM label as domain-wide. SES stores it on the domain identity, so a second sender reuses the existing label instead of moving it for every sender.
• Scoped API keys and MCP OAuth can call the four actions only while the flag is on for the caller. With it off, they still get "This action does not support personal API key access". The check reads (integration:read). Verify, apply-url and email update write (integration:write), because verify and apply-url call SES.
• Session verification, sender updates, discovery and apply-url keep existing access, including email and proxy links while the agent flag is off.
• The four actions now declare request, query and response schemas. URLs and success responses do not change.
• Validation errors keep their message text, but now carry an attr. context and provider_endpoint errors on apply-url can come back together instead of one at a time.
• The generic integrations-create stays disabled. It accepts an untyped config for 30+ kinds, including credentials. integrations-email-create reuses the same POST integrations/ route with kind: email injected and the typed email config as its only input.
• The MCP email tools default provider to ses. The API still requires it.
• The MCP Domain Connect tool only builds URLs for email senders. It always sends context: email, hides context and proxy_record_id, and requires integration_id. The API accepts reverse proxy context only for sessions. Scoped API keys and OAuth receive 403 before signing a proxy URL.
• email/verify and PATCH email on a non-email integration now return 400 instead of 500.
• Domain Connect apply-url now returns 404 for an unknown or non-email integration_id, and for an unknown proxy_record_id from a session. Scoped proxy requests receive 403. Before, a stale id returned 500, and a non-email id returned a "try again later" 400 and reached error tracking.
• With the flag on, PATCH email with a different address returns 400. With it off, it still returns 200 and keeps the old address. The check ignores case, so a sender stored with capitals before addresses were lowercased can still be edited.
• proxy_record_id is now a UUID field, so a malformed value returns a 400 with attr instead of a 500.
• A new sandboxed eval suite, eval_email_domain_setup, checks that an agent with the skill sets up a domain correctly. Details below.
• An MCP codegen fix: param_overrides.<field>.required now also works together with input_schema. Before, it was silently ignored.
• Mechanical: regenerated OpenAPI, frontend types, MCP Orval schemas and tool definitions.

Evals

The suite runs a real agent with the skill against the real email API over MCP. SES, DNS and Domain Connect are simulated per case, so each case decides which records already exist.

• Scorers grade outcomes, not the order of tool calls: the senders in the project and their verified state, the exact apply URL, the record values, and bounded verify calls. LLM judges check the hand-off and error messages against a reference list of records.
• Every run uses new sending domains. PostHog refuses a domain another organization uses, and each case runs in its own organization in a kept database.
• The suite rejects --trials above 1 for the same reason. Trials of a case share its domain.
• The eval harness now turns on workflows-email-domain-agent-setup for its MCP server, so evals can reach the tools.

Nothing user-visible changes in the web app.

• workflows-email-domain-agent-setup: Default-off server guard for new token authorization; MCP tools use the same flag. Existing session apply-url access remains available.

• Scoped credentials only support email Domain Connect. Session email and proxy access does not depend on the agent flag.

How did you test this code?

• Current test seams: parameterized HTTP requests with PAK, OAuth and session credentials, both flag states, and a mocked signing boundary.

• Before: scoped proxy credentials reached signing, and flag-off sessions received 404. After: scoped proxy calls return 403; baseline sessions succeed.

• Local proof: agent-final-backend-green.log, agent-final-mcp-tests.log, agent-final-openapi-green.log, agent-final-typecheck.log and agent-final-preflight.log.

• 62 focused backend HTTP/domain/scorer/simulated-eval tests passed; domain-109703-gate-green.log.

• The session flag-off test now preserves baseline 200 behavior. PAK/OAuth proxy tests failed before the fix and pass with 403.

• 268 MCP generator/apply-url/tool-filtering tests and MCP TypeScript passed; domain-109703-mcp-tests.log and domain-109703-typecheck.log.

• Full OpenAPI regeneration and skills lint passed; domain-109703-openapi.log and domain-109703-skills.log.

• Strict ci:preflight passed; domain-109703-preflight.log.

Release status

Fork handoff only. This task does not activate rollout flags. Deployment flag state was not inspected. Upstream CI and the unavailable fresh Opus review remain handoff gaps.

• This change is behind a feature flag and is not available to users

Docs update

Updated the existing OAuth development guide with email-only Domain Connect access for scoped credentials and baseline session access.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, Claude Opus 5.5 (claude-opus-5-5[1m])

• Started by Michael's orchestration thread as "PR B" of a settled design for the email domain redesign. PR A owns the wizard and the GET email/status endpoint; whoever lands second reruns hogli build:openapi.
• Skills invoked: /improving-drf-endpoints, /implementing-mcp-tools, /writing-tests, /writing-skills, /writing-evals, /running-ci-preflight, /reviewing-with-coderabbit, /writing-pr-descriptions, /tdd, /qa-swarm.
• Codex review (GPT-6.1 Sol) found 3 issues, all fixed: the create tool still required kind, the update tool left config optional, and the skill reused a sender with a different address on the same domain.
• Adversarial reviews by Fable 5.1 and GPT-6.1 Sol found 8 issues, all fixed: Domain Connect duplicating DMARC (now handled by fix(workflows): keep an existing dmarc record when auto-configuring email dns PostHog/posthog#109695), the per-sender MAIL FROM assumption, re-create resetting verification, SPF records that verify never checks, the @ hostname missing from the help text, wrong permission levels in the skill, a 500 on non-email ids, and a test that only ran as admin and never read bodies.
• Michael asked for evals of the skill. Adversarial reviews of the eval suite by Fable 5.1 and GPT-6.1 Sol found 11 distinct issues, 10 fixed: domains shared across runs and trials, scorers that passed an unverified sender, a shortened apply URL, incomplete records or an SPF include after all, a substring check that failed a correct DMARC answer, a shared discovery cache, an avoided-tool check on a tool the agent cannot see, a judge that penalized the SPF reminder the skill asks for, a member case that never checked the created sender, and DNS state the prompts describe but nobody seeded. Not changed: scorer tests under evals/test/, which follows the notebooks product.
• Merged master twice to clear staleness advisories. The second regen changed only proxy_record_id to a UUID in the generated zod schema.
• A second QA round (six GPT-6.1 Sol lenses and Fable 5.1) found 9 distinct issues. Fixed: the apply-url errors above, a scorer that ignored case in TXT values, a private scorer entry point in tests, the record type label that left out SPF, and three skill gaps (a DMARC lookup timeout read as a missing record, no source for the sender name on update, a member continuing after a 403). Rejected: per-team tenant scoping in the SES simulation, which is documented and exercised on purpose.
• A third QA round found 2 more, both fixed: the silent address change on update, and a missing check on the records handed over in the taken MAIL FROM label case.
• A fourth QA round found a type-annotation gap, fixed, and one open question: adding apply-url to the write scope also opens its reverse proxy branch to integration:write keys. That earlier decision is superseded: the current fix rejects scoped proxy credentials and retains baseline session proxy access.
• CodeRabbit's security review called this PR moderate risk. Its DMARC finding was already fixed by fix(workflows): keep an existing dmarc record when auto-configuring email dns PostHog/posthog#109695, and its MAIL FROM finding is fix(workflows): keep one mail from label per email sending domain PostHog/posthog#111367. Michael then asked for the backend behind the flag too, so the PR can merge with no behavior change for API callers. Checking the risk found the mixed-case regression in the address check.
• A Codex (GPT-6.1 Sol) adversarial review of the flag gate found one issue. Rejected: it reported the DMARC lookup in apply-url as new flag-off behavior, but that code is fix(workflows): keep an existing dmarc record when auto-configuring email dns PostHog/posthog#109695, already on master. Its other point, that the request schemas are not flagged, is by design and listed under Release status.
• CodeRabbit CLI: skipped, signed out with no person available during the run.
• Duplicate search: no open PR covers this.
• Follow-up: once PR A's status endpoint lands, the skill should poll it instead of verify.
• Follow-up: #111367 keeps the domain's MAIL FROM label in step across senders. Until it lands, changing the label on one sender gets undone when another sender on the domain verifies. Michael asked for it to go behind the same flag, so turning the flag on enables both. Keep the flag off until both have deployed. The address check now runs in the API view, so this PR no longer edits update_native_integration.
• Follow-up: #111366 applies MCP exclude_params per tool. Today integrations-email-create removes kind from the shared create body, which only matters if the generic integrations-create tool is enabled.
• Nothing in this PR comes from customer material.

Modernization used Codex, gpt-6.1-sol. Skills: modernize-pr, pr, writing-pr-descriptions, writing-voice, merge-risk, unslop. The source replays onto master at b75567aca6cce9f0499e882463fc7e71d65a064c. Original authorship and prior evidence remain above. The required current Opus review could not run because the Claude API rate limit blocked it.

Current review fixes and final upstream replay used Codex, gpt-6.1-sol. The fresh cross-family review remains blocked by the known Claude rate limit.